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I.  SUMIiARY, 

The  triangle  test  has  been  used  very  frequently  in  sensory  difference 

testing.  Attempts  have  been  made  to  augment  the  basic  information  from  triangle 

tests  with  scores  on  degree  of  difference  between  the  sample  selected  as  the 

variant  and  the  remaining  two.  In  this  paper  a  mathematical  model  has  been 

developed  to  permit  formal  utilization  of  the  degree  of  difference  scores. 

It  is  assumed  that  a  stimulus- response  scale  exists  and  that  tv/o  standard 

samples  and  one  variant  sample  evoke  responses  x^,  x^  and  y  on  this  scale.  These 

2 

responses  are  taken  to  have  independent  normal  distributions  with  variances  a  and 
means  zero  for  the  x-variates  and  mean  jj.  for  the  y-variate.  Conditional  distribu¬ 
tions  for  degree  of  difference  scores  are  obtained,  the  likelihood  function  for  N 
independent  trials  of  the  triangle  test  is  developed,  and  a  procedure  for  testing 
that  |i  =  0  is  given. 

The  procedure  considered  is  designated  as  the  modified  triangle  test  and 
illustrations  of  its  use  are  shown. 
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II,  INTRODUCTION. 

Sensory  difference  tests  are  used  frequently  in  many  situations  involving 
the  subjective  responses  of  individuals.  These  tests  may  be  used  for  the  main¬ 
tenance  of  quality^  the  detection  of  adulterants^  the  substitution  of  ingredients, 
the  screening  of  product  variants,  the  measurement  of  sensitivities  of  individuals, 
and  the  selection  of  test  panels.  The  triangle  test  is  perhaps  the  most  frequently 
used  of  the  simple  sensory  difference  tests  and  is  the  subject  of  this  paper. 

There  has  been  a  certain  amount  of  confusion  in  the  literature  on  the 
choice  of  an  appropriate  sensory  difference  test.  Experimental  comparisons  have 
been  made,  for  example,  by  Liebmann  and  Panettiere  [1957],  Mathematical  and 
statistical  comparisons  have  been  made  by  Hopkins  and  Gridgeman  [1955],  Radkins 
[1957],  Ura  [1960]  and  Bradley  [1958,  1963].  These  comparisons  suggest  the  use 
of  the  triangle  test  in  most  situations  v;here  a  simple  procedure  is  required. 

Efforts  have  been  made  to  modify  the  traingle  test  in  order  to  obtain 
improved  performance  in  the  detection  of  differences.  Much  of  this  work  appears 
to  be  unpublished  but  an  example  is  given  by  Mahoney,  Stier  and  Crosby  [1957], 

The  basic  triangle  test  can  be  discussed  in  terms  of  discrimination  between  a 
standard  product  and  a  variant  from  that  standard.  A  group  of  N  respondents  are 
each  presented  a  traingle  test.  The  presentation  involves  two  samples  of  the 
standard  and  one  of  the  variant,  all  unidentified,  and  it  is  the  task  of  the 
respondent  to  select  the  variant  sample  from  the  three  samples  on  the  basis  of  the 
sensory  characteristic  under  study.  In  the  modifications  of  the  triangle  test 
considered,  an  additional  task  is  imposed  on  the  respondent.  For  the  modified 
test,  the  respondent  not  only  must  select  the  variant  sample  but  must  also  “score” 


the  "degree  of  difference"  present.  This  is  usually  done  on  the  basis  of  a  crude 
scale  implying  that  the  difference  is  "slight/*  "moderate^"  "easily  detectable," 
"extreme,"  etc.  In  past  use  of  the  modified  triangle  test,  it  appears  that  the 
scores  provided  have  been  used  largely  as  a  qualitative  check  on  the  test  rather 
than  for  specific  analysis. 

It  is  the  purpose  of  this  paper  to  provide  a  method  of  analysis  for 
modified  triangle  tests.  In  providing  this  method,  certain  assumptions  are  made 
that  require  further  study  but  it  is  believed  that  a  useful  technique  has  been 
developed  to  improve  sensory  difference  testing. 

III.  FORMULATION  OF  THE  PROBLEM. 

A  basis  for  the  analysis  of  modified  triangle  tests  has  been  provided 
by  Bradley  and  Ura  in  the  papers  cited.  Furthermore,  that  work  indicated  that 
the  resulting  analysis  may  not  be  too  sensitive  to  the  assumptions  made. 

Following  Bradley  [1963]  suppose  a  conceptual,  sensory-difference, 

stimulus-response  scale  for  sensory  sensations  of  respondents  in  difference 

testing.  Assume  that  responses  to  the  standard  samples  are  normally  distributed 
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with  mean  zero  (the  origin  of  such  a  scale  is  arbitrary)  and  variance  a  .  The 
responses  to  the  variant  are  likewise  assumed  to  be  normally  distributed  with 
variance  but  with  mean  |j..  All  these  stimulus  responses  are  taken  to  be 
stochastically  independent.  Let  the  two  responses  to  the  standard  be  and  x^ 
and  the  response  to  the  variant  be  y.  Whether  or  not  the  respondent  correctly 
selects  the  odd  or  variant  sample  from  the  three  is  assumed  to  depend  on  the  con¬ 
figuration  of  the  responses  x^,  x^  and  y.  Twelve  possible  configurations  may  be 
obtained  and  are  designated  by  •  *  *  >^^2  follows: 
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The  first  four  of  these  configurations  lead  to  correct  selection  on  the  triangle 
test  and  the  score  on  degree  of  difference  is  taken  to  approximate  to 
^  “  |y  -  last  eight  of  these  configurations  lead  to  incorrect 

selection  and  the  score  on  degree  of  difference  measures  either  W  =  |x^-%(y  +  X2) | 
or  W  =  -  %(y  +  ^1^1*  following  work  it  is  assumed  that  R  and  W  have 

distributions  based  on  the  indicated  functions  of  normally  distributed  variates 
even  though  observed  values  of  R  and  W  are  imperfectly  measured  on  discrete 
scoring  scales. 

At  this  stage  note  that  data  from  N  modified  triangle  tests  will  appear 
as  scores.  ,  m  +  n  =  N,  in  some  order  with  m  and  n  themselves 

random  variables.  On  the  basis  of  such  data_,  a  test  of  significance  is  developed. 
The  null  hypothesis  is  that  [1=0^  there  is  no  difference  between  standard  samples 
and  variant  samples  on  the  basis  of  the  sensory  attributes  under  study.  The 
alternative  hypothesis  is  that  i-i  ^  0.  The  hypothesis  that  p  =  0  is  equivalent, 
of  course,  to  the  more  standard  statement  that  the  probability  of  correct 
selection  of  the  variant  sample  is  1/3,  a  selection  based  on  chance  alon.e.  If 
p^  is  the  probability  that  correct  selection  be  made,  p^  is  related  to  }u/a  as 
given  by  Bradley  [1963]: 
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2/0  2 
M-  /3a 


h,  ra  +  i) 


(1  +  u)^ 


(Bradley  and  Ura  have  given  tables  associating  values  of  with  values  of  h/a 
The  next  section  deals  x^ith  the  problem  of  obtaining  distributions  of  R  and  W 
based  on  the  assumptions  made.^ 

IV.  DISTRIBUTIONS  OF  DIFFERENCE  SCORES. 

The  joint  likelihood  function  of  built  up 

from  the  conditional  distributions  of  R  and  W  given  the  appropriate  configurations 
from  Configuration  leads  to  a  value  of  R  and  we  develop  now  the 

conditional  probability  density  function  of  R  given  in  some  detail. 

The  conditional  joint  probability  density  function  of  and  y  given 

C  ^  on  the  avssumption  of  the  normal  distributions  and  independence,  is 


f  (x^,x2,y  I 


?  "  3/2 
F{G.) 


-  -1-  +  (y  -  [i)^]/2a^ 


X-  <  x^  <  y,  -  X,  <  y  -  x„,  -  w  <  x-,  x^,  y  <  <», 

iZ  ^JL  Z  ““iZ  ' 

\'7here  P(C^)  is  the  probability  of  occurrence  of  configuration  C^.  It  is  clear 
that  (2)  follows  from  the  joint  unconditional  distribution  of  x^,  x^  and  y  which 
differs  from  (2)  only  in  the  emission  of  the  division  by  ^nd  the  first  two 

restricting  inequalities  follox<ring  (2).  An  orthogonal  transformation  to  new 
variables  r,  s  e.nd  t  is  made  with  r  -  (2y  -  x^  -  x^)/-/^^,  s  =  (y  + 
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and  t  =  (x^  -  x^)/72  ,  Then  the  joint  probability  density  for  the  new  variables  is 


e-  -  (%r/V6)  -  (2^s/^)  +  ^3^ 

P(cp 

0  <  t  <  (r/73)  <00^  ^  S  ^  ^ 

It  is  sseu  that  r  is  related  to  the  required  R  and  integration  with  respect  to  s 
and  t  leads  to  the  marginal  conditional  distribution  of  r, 


f(r|cp 


P(C3^) 


I(r/a  -/S) 


{r^  -  (4^ir/76)  +  (2^^/3)}/2a^ 


(4) 


0  <  r  <  00,  In  (4)  the  function  I  is  the  incomplete  standard  normal  integral, 


1(a)  =  (27t)'^ 


du« 


0 


(5) 


Since  R  =  |y  "  +  x^)!  and  r  >  0,  R  =  76  r/2  and  the  required  conditional 

probability  density  function  for  R  is 


°  I(^  R/3a)  ,  (6) 

0  <  R  <  00. 

It  follov7S  from  symmetry  that  f(R|C2)  has  the  same  form  as  (6)  and  indeed 
P(C2)  =  P(C^).  fCRjc^)  and  f(Rjc^)  have  the  form  of  (6)  but  with  |i  replaced  by 
“|i.  Accordingly,  we  may  compound  these  conditional  probability  density  functions 
to  obtain  the  probability  density  function  of  R  given  correct  solution  in  the 
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triangle  test.  Thus 


f(R|A)  =  Y 


P(C,) 


f(R|cp 


I(  -/2R/3a) .  cosh  (2f:iR/3a  )  e 


2  2  7 

^  .  “(R  +  ^  )/3a^ 


0  <  R  <  oo>  =  E  shown  in  (1). 

We  require  also  the  conditional  distribution  of  W  for  configurations 

joint  conditional  distributions  of  ^  x^ritten  down 

as  for  (2).  We  consider  in  a  little  detail;  now  the  orthogonal  transformation 
has  r  =  ~  ^2  "*  ®  before  and  t  =  (y  -  x^)/*/?.  The  insertion  of  W 

comes  since  W  =  “  t/3  r/  /2.  Steps  similar  to  those  for  f(R|c^)  lead  to 


fWiCj)  .  ,-(w-^)W 


J  _  Ii  V 

^  V  3a  ^  2  +  ^ 


with  0  <  W  <  CO. 

From  symmetry,  f(w(c^)  has  the  form  (f3)  and  f(w|c^)  and  f(w|Cg)  have  the 
form  of  (8)  x^rith  iJi  replaced  by  -[i  and  simplified  since  I  (-a)  =  -  1(a).  For 
f(w|c^),  the  transformation  is  the  same  as  for  (8)  but  the  boundaries  of  the 
region  differ;  however,  the  same  steps  give  us 


fmjc  )  =  -  \  T 

PCCg)  e  z  i 


72W 

aV2 
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0  <  W  <  “•  f(wjc^Q)  has  the  form  of  (9)  and  f(w|c^^)  and  f(w|Gj^2)  the  form 

of  (9)  with  \i  replaced  by  -li. 

The  marginal  conditional  distribution  of  W  given  that  the  selection  in 
the  triangle  test  is  incorrect  (denoted  by  A)  is  available  now  as 


PS  \ 


L  V  ^72 


+  II 


•/2W 

3a 


.72: 


0  <  W  <  00  ,  p-  =  1  -  p^. 


V.  ESTIMATION  AND  DIFFERENCE  TESTING. 


With  the  mathematical  developments  of  the  preceding  section^  we  may  now 
turn  again  to  the  problem  of  sensory  difference  testing.  Suppose  as  before  that 
N  triangle  tests  yield  scored  differences^  ,  W,  .....W  ,  m  +  n  =  N  and 

note  that  this  implies  ra  correct  selections  of  the  variant  sample  and  n  incorrect 
selections.  The  likelihood  function  may  be  written  as 


L(R,,. 


I  .  ^  R  ) 


W. 


)  =  L 

^  n 


m  In 

n  f(R,  |a)  p. 
i=l  ^  J  A 


n  __ 

*T[  f(W.|A) 
i=:l  J 


Use  of  (7)  and  (10)  with  possible  simplifications  gives 


^9^  9  9  9  9 

L  =  4^  a~^  e  "  ^  ^  \  ^  e  '  +  n)/12a^ 

m 

X  ■  IT  cosh  (2iaR./3cr^)  IT  cosh  ([i  W./3a^) 
i-1  ^  j-1  ^  . 


X 


m 

n  I(  72  R./3a) 
i=l 


JtL. 

a72 


+  I 


3a 


,(10) 


(11) 


9. 


While  (11)  is  a  difficult  likelihood  function^  we  shall  proceed  to  develop  itera¬ 
tive  procedures  for  maximum  likelihood  estimates  of  \i  and  a  and  to  develop  the 
likelihood  ratio,  test  of  the  null  hypothesis,  l-i  =  0,  versus  the  alternative, 

H  :  0. 

a 

The  procedure  that  we  have  adopted  is  to  maximize  InL  with  respect  to 
9=5  /a  and  a,  the  reparameterization  yielding  minor  simplifications.  Then,  with 
k  independent  of  9  and  a. 


m  n  2  2  2 

InL  =  k  -  N  Ina  -(ER.  +  EW.)/3a  -  (4m  4-  n)  0  /12 

1  1  ^ 


+  E  In  cosh  (20R./3a)  -f  E  In  cosh  (9W./3a) 

1  J 


m  n 

+  E  In  I  (V^R./3a)  +  Z  In 
1 


3a  ^  J  V  3a  ^ 


(12) 


To  simplify  notation,  we  now  let 


V2Wj  N 

^  i  +  -  )  and  I  .  =  I  (  -o —  “  - 


(13) 


Similarly,  if 


i  ^  /2 

<I>(a)  =  -  e  , 

42t\ 


Vzw.  N  /  ■ 

1>x  •  =  ®  i  +  -  j  and  0  .  =  $  (  -r - - 

+>3  V  3a  ^  y  -,j  V  3a  ^ 


(14) 
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To  maximize  InL  (and  L)  and  to  obtain  0  and  o  the  maximum  likelihood  estimators 

/\  /N 

of  Q  anduoC^i  =  a  6)^  the  partial  derivatives  of  InL  with  respect  to  9  and  a 
are  set  equal  to  zero.  The  resulting  equations  are 


fl(0>cT>  =  a  ^  InL 

m 

=  -  90  (4m  +  n)/6  +  ^  (2R^/3)  tanh  (20R^/3a) 

i=*l 


+  ^  (W./3)  tanh  (ew./3a) 
j=l 


+  (a/i/2)  y  ($  -  $  ,)/(!  +  I  )  =0 

/-J 


and 


fp/e^a)  =  a  -gj  InL 


=  -  N  +  (2/3a^)( 


’j 


i=l  j=l 


y  (2eR,/3cr)  tanh  (2eR^/3CT) 


i=l 

n 


-  ^  (0Wj/3a)  tanh  (9\lj3a) 


j=l 

m 


y  (V2  R^/3a)  $  (72  R^/3ct)  /  I  (>^  R^/30) 

Z-j 


j=i 


(15) 


11. 


-I 


(72  W./3ct)((5  .  + 

J  J 


I  ,)  =  0. 
J 


(16) 


These  equations  are  solved  iteratively  (and  quite  easily  with  an  electronic 
computer).  The  usual  procedure  is  followed^  namely  to  use  initial  estimates 
6  and  a  ,  solve  the  linear  equations, 

f.  (e  ,  a  )  +  ^fll  +  Aff  =  0  , 

°  °  ae  le  Sa  ie  ,a  (17) 

!  O''  O  O*^  O 

and 

)  -1  Afi  +  Aa^i  =  0  ,  (18) 

^  ^  S©  I0  5a  is 

0^0  J  o 


for  and  Aa^  and  obtain  6^  =  0^  +  A0  and  =  a^  +  Aa  as  improved  estimates. 

The  procedure  is  repeated  until  satisfactory  convergence  to  Q  and  a  is  obtained. 

The  required  derivat ives’^ f or  (17)  ^ncj-  (18)  are: 

m 

-  a(4in  -I-  n)/6  +  (2R^/3a)^  sech^  (20  R^/3a) 

i=l 


n 

-fa)  (W./3a)^  sech^  (0W./3n) 

A-j  J  j 

j=l 


(a/2) 


I 

j^l 


n 


I 

j=i 


+  4  + 


J 


'+^  j 


4)^ 
y  J 


+  (a/2) 


12. 


af 


ha 


=  -  0  (4m  +  n)/6  -  (0/a^)  ^  (2R./3)^  sech^  (20  R^/3ct) 

i=l 


(0/a^)  y  (W./3)^  sech^  (0  W./30) 
^  J  J 

j-1 

n 


(1/V2)  y  (4  ,  -  t  )/(i 
^ 
j=i 

n 

)  (W  /3a)(0^  -  )/(I  +  I 

j=l 


.  y 


5^2  =  |£l  -  f  /a  , 

Bo  Ba 


and 

Ba 


m  n 


=  (l/a) 


-  (4/3a^)(  +  y^j)  +  y  (20R./3a)  tanh  (20R^/3a) 


i=l  j=l  i=l 

n 


y  (20R./3ct)^  sech^  (20R./3a)  +  ^  (0Wj/3a)  tanh  (0Wj/3a) 


i=l 

n 


j=l 


y  (0Wj/3a)^  sech^  (0Wj/3a) 


j=l 

m 


)  (y2R,/3a)  $  (■/2R./30)/l(-/2R./3a) 

L _ I  1  1  1 


i=l 


(19) 
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m 


+  )  (72R./3a)^  i^'(72R./3a)/l(72R./3a) 

L I  i  11 

i=l 

m 

-  )  (72R./3a)^  3)^(V^R./3a)/l^(v^R./3CT) 

L I  1  11 

i=l 

n 

+  )  (,-Pbi  I3a)  (0,  .  +  <5  J/(I,  .  +  I  J 

4 — I  J  'yJ 

j=i 

n 

+  y  (V2W  /30)^  ($'  +  )/(I  .  +  I.  J 

j=l 


^  (y2W./3a)^  (3.^  ,  +  $_  /  (I^  ,  +  I_  y 


+y2 


j-1 


+y2  -jrJ^ 


Note  that  if  the  information  matrix  leading  to  asymptotic  variances  and  covariance 


of  0  and  a  is  required^  this  may  be  computed  easily  at  the  last  stage  of  iter¬ 
ation  since  5^1nL/S0^  =  (df^/^0)/a^  S^lnL/S0Sa  «  [(Sfj^/Sa)  -  (f-^/o)]/o  y  and 
S^lnL/5a^  =  [(Sf^/Scr)  -  (f^/a)]/cT. 

Initial  values  0^  and  may  be  obtained  in  various  ways.  Examples  con- 

/\  /V 

sidered  have  shown  quite  rapid  convergence  to  0  and  a  even  when  0^  and  are 
quite  poor  first  approximations.  An  easy  procedure  for  obtaining  0^  is  to  take 
m/N  as  an  estimate  of  p.  and  to  read  0  =•  a  /a  from  Table  I  of  the  paper  by 

Bradley  [1963].  A  satisfactory  value  for  may  be  obtained  by  examination  of 
the  range  of  the  and  but  an  additional  method  is  given  in  the  appendix. 
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When  Q  and  a  have  been  obtained;  the  maxiinum  of  InL  is  calculated. 

In  order  to  develop  the  required  likelihood  .ratio  test;  we  need  also  to 
obtain  the  maxiraura  likelihood  estimator  of  a  given  that  [i  -  0  or  0  =  0.  Careful 
computer  programming  can  include  this  case  in  the  general  estimation  procedure  or- 
a  simpler  program  may  be  developed.  When  0=0;  InL  in  (12)  is  considerably 
simplified  and  the  equation  to  be  solved  is  =  0.  This  solution  is  devel¬ 

oped  iteratively  again  from  the  linear  equation  in  Aa  from  (18); 


f2(0,a),;.;f2(0,a^)  +  Ag  ^^2 

Bo 


0;  a 


(20) 


with  Sf^/Sa  at  (0;a^)  obtainable  from  (19). 


If  we  denote  the  value  of  a  that  maximizes  L  given  0  =  0  by  O;  the 
maximum  of  InL  is  obtained  through  substitution  of  (0;a)  for  (0;  o')  in  InL. 

The  usual  asymptotic  theory  is  used  for  the  test  of^signif icance.  If  X 
is  the  likelihood  ratiO; 


-  2  in  X  =  -  2[1 


A  ^ 

jo^ff 


InL 


§/d 


(21) 


is  taken  to  have  the  X  -  distribution  with  one  degree  of  freedom. 


VI.  NUMERICAL  EXAMPLES. 

Data  for  N  =  44  modified  triangle  tests  together  with  scored  ’Megree  of 
difference"  are  given  in  Table  1,  The  scoring  scale  used  is  no  difference  :  0; 
very  slight  difference  :  2;  slight  difference  :  4;  moderate  difference  :  6;  large 
difference  :  8;  and  very  large  difference  ;  10*  The  triungle  tests  compare  an 
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TABLE  1. 

SCORES  ON  DEGREE  OF  DIFFERENCE  ON  CEREAL  TESTS^ 
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experimental  cereal  with  a  control  cereal  and  some  triangle  tests  involved  two 
samples  of  the  control  cereal  with  one  of  the  experimental  and  some  tests  used 
one  control  sample  and  two  experimental  samples.  Note  that  m  =  20  and  n  -  24. 

/S 

To  obtain  G  and  o,  the  iterative  procedure  of  equations  (17)  and  (18) 
was  initiated  with  arbitrarily  selected  values^  0^  =  1,  *  2.5.  The  IBM  709 

computer  was  used  with  instructions  to  terminate  iterations  when  successive 
values  of  9  and  a  agree  to  the  seventh  decimal  place.  Successive  values  of  G^a 
and  InL  to  four-place  accuracy  are  given  in  Table  2.  To  obtain  "a  given  0=0^ 
the  initial  value  of  was  again  2.5^;  values  to  four-place  accuracy  with  values 
of  InL  in  this  case  also  are  given  in  Table  2.  The  test  statistic  is  computed 
from  (21), 

-  2  InL  X  -  2  [-116.1963  -  (-114.1305)]  =  4.1316 
2 

and  comparison  with  X  -tables  with  one  degree  of  freedom  indicates  that  this  value 

comes  at  approximately  the  .044  level  of  signigicance. 

If  the  usual  triangle  test  had  been  used,  a  one-sided  test  of  significance 

would  have  been  performed  with  the  null  hypothesis  being  1/3  and  the 

alternative  being  H  :  p .  >  1/3.  Using  N  -  44^  m  »  20  and  the  normal  approximation 
a  ^ 

to  the  binomial  with  continuity  correction,  we  have 

^  ^  m  -  (N/3)  +  1/2  ^  3(20  -  14.6667  -  .5)  ^  ^ 

^ir  ■ 

with  significance  level  .061.  The  confidence  interval  on  p^  with  confidence 
coefficient  .95  by  the  usual  methods  is  (.307,602). 
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TABLE  2 

PARAMETER  VALUES  AND  InL  FOR  SUCCESSIVE  ITERATIONS 


Iteration 

Alternative  Hypothesis 

Null  Hypothesis  0=0 

e 

0 

InL 

0 

InL 

0 

1 

2.5 

2.5 

1 

1.2581 

1.8825 

-  114.8550 

2.4859 

-  116.1963 

2 

1,1186 

2.0639 

-  114,1520 

2.4860 

-  116.1963 

3 

1,0820 

2.1054 

-  114.1306 

2,4860 

-  116.1963 

4 

1.0809 

2.1069 

-  114.1305 

5 

1.0809 

2.1069 

-  114.1305 

---- 
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A  confidence  interval  and  estimate  of  may  be  obtained  from  the  modified 
triangle  test.  We  have  our  estimate  of  6  as  1.0809  from  Table  2.  Using  Table  I 
of  the  reference^  Bradley  [l963]^  we  equate  6  with  [x/o  in  column  1  and  interpolate 
in  column  2  to  find  p^  =  ,431.  A  confidence  interval  on  Q  may  be  converted  into 
a  confidence  interval  on  p^.  The  computer  program  produces  the  required  partial 
derivatives  of  f^  and  f^  so  that  the  matrix^ 


-  5^1nL 
So  5a 


-  5^1nL 
SoSa 


S^lnL 


/ 

18.4175  20.2982 


20.2982 


43.8007 


may  be  easily  computed.  The  inverse  of  this  matrix^  from  large-sample  theory  on 
maximum  likelihood  estimators^  gives  the  variance- covariance  matrix  for  0  and  a 
This  matrix  is 

.11098  -  .05143 

-  .05143  .04666 

so  that  (0-0)  may  be  taken  as  normal  with  zero  mean  and  variance  .11098.  The 
,95-confidence  interval  on  6  is  evaluated  easily  as  (.4280;  1.7338)  and  the 
corresponding  interval  on  p^  is  (.350;  ,552). 

Harmon  [1963]  used  a  made-up  example  wherein  the  values  of  x^  and  x^  were 
taken  from  a  table  of  random  normal  variates  with  mean  zero  and  variance  unity 
whereas  the  values  of  y  were  chosen  similarly  but  from  a  population  with  mean 
1.5  and  variance  unity.  In  his  exanple;  N  =  20;  m  =  9  and  n  «  11,  He  found 
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A 

Q  =  1.7097  and  a  =  .9699  under  H  and  0'  =  1.3620  under  H  •  For  this  example, 

a  o 

2 

X^  =  -  2  lnX==  4.358  for  the  modified  triangle  test  and  the  normal  deviate 
u  =  .870, for  the  conventional  binomial  triangle  test.  The  estimates  of  were 
respectively  .547  and  .450  while  the  true  value  of  p^  for  the  populations  sampled 
is  .5065. 

The  example  worked  by  Harmon  appears  to  give  satisfactory  estimates  of  the 
known  parameters  of  the  population  sampled  and  to  suggest  that  improved  test 
power  may  be  attributed  to  the  modified  triangle  test.  The  example  of  Table  1 
seems  also  to  be  satisfactory  in  that  higher  significance  is  obtained  with  the 
modified  triangle  test  in  comparison  with  the  simple  triangle  test  and  a  shorter 
confidence  interval  is  obtained  on  p^.  One  cannot  judge  the  merits  of  the  modified 
triangle  test  procedure  on  the  basis  of  one  or  two  examples.  Extended  experience 
or  mathematical  investigations  should  show  that  the  modified  triangle  test  is 
more  powerful  than  the  simple  triangle  test  in  the  detection  of  differences  in 
sensory  testing. 

VII.  INDETERMINATE  FORMS  IN  COMPUTATIONS. 


The  theory  of  the  preceding  sections  is  based  on  a  model  that  attributes 
probability  zero  to  the  occurrence  of  zero-values  of  or  W^.  However,  in 
practice  as  in  Table  1,  zero-values  do  occur  from  the  discreteness  of  scoring 
scales  used.  Unless  consideration  is  given  to  this  in  computer  programming, 
difficulties  will  occur. 

If  an  R.  or  W.  is  zero,  indeterminancies  occur  in  f _ ,  f^,  Sfi/S0,  Sf- /So, 

1  j  ^  1*^  2'"  1  I  ^ 

5f2/S0  and  Sf^/So.  Programming  should  be  done  so  that,  if  or  is  zero,  the 
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appropriate  limits  replace  the  indeterminate  forms.  The  necessary  limits  are  as 
follows : 


lim 

xil)(x)/l(x)  =  1, 

(22) 

x-^ 

lim 

x->0 

(23) 

lim 

X“*0 

+  ij  =  - 

(24) 

lim 

x->0 

i_f  =  -  Q^n, 

(25) 

lim 

x-~>0 

[$;  +  $:']/[i^+  I.]  -  1, 

(26) 

lim 

x-^ 

x[$J  -  $?]/[!+  +  i.]^  =  - 

(27) 

lim 

X“+0 

x[<5^  -  +  i_]  =  - 

(28) 

lim 

x^  Ili^(x)/I(x)  =  0, 

(29) 

X“^ 

lim 

x^  (I>^(x)/I^(x)  =  1, 

(30) 

X“»0 

lim 

x->0 

x^[$'  +  0 ']/[!,  +  I  ]  =  0, 

(31) 

lim 

x-»0 

+  0  fill.  +  I  ]^  =  1. 

"T  -  T*  ** 

(32) 

Rules 

must  also  be  developed  in  computing  the  tnaximum  of  InL. 

Subroutines 

forrcomputers  for  logarithms  usually  write  InO  «  0  and  this  is  not  satisfactory. 

Consider  InL  in  (12)  and  note  that  to  calculate  -21nX  in  (21)^  some  rules  for 
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lim  [In  I(x/a)  -  In  I(x/a)]  =  In  "a  -  In  a 
x-»0 

and 

lira  {ln[l(^  +  -  )  +  !  (?-“)]-  In  2l  (x/a)} 

x-^q  ^  4i  ^  n 

=  Ino^  “  Ina  -  t  0^ 

4 

must  be  devised.  It  is  satisfactory  to  program  the  computer  so  that 


In  I(x/a) 


x=0 


Ino 


(33) 


and 


4 


x-O 


1  1  cP- 

Ina  “70. 
4 


(34) 


These  rules  seem  to  be  the  simplest  workable  ones  and  yield  the  proper  value 
of  “  2  In  X;  note  that  if  zero- values  of  and  are  replaced  by  a  small  number^ 

say  .001,  different  values  of  InL  will  be  obtained  than  with  the  above  rules  but 
approximately  the  same  value  of  -2  InX  will  result.  These  rules  have  been  used 
in  the  first  example  of  the  preceding  section. 


VIII.  DISCUSSION. 


The  procedures  for  the  modified  triangle  test  hold  the  possibility  of 
improved  efficiency  in  sensory  difference  testing.  It  is  true  that  numerical  work 
is  greatly  increased  in  the  analysis  in  comparison  with  the  simple  triangle  test^; 
however,  electronic  computers  are  now  widely  available  and  once  a  computer  program 
is  developed^  ^plication  of  the  modified  triapgle  test  is  easy.  The  example 
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given  in  this  paper  is  shown  in  sufficient  detail  to  check  on  programming  and  may 
be  used  for  this  purpose. 

Further  investigation  of  the  modified  triangle  test  could  be  useful.  Con¬ 
sideration  of  power^  either  by  asymptotic  theory  for  large  N  or  by  Monte  Carlo 
techniques^  would  be  desirable.  The  effects  of  discreteness  in  degree-of- 
difference  scores  in  comparison  with  the  model  used  should  not  be  serious  but 
might  be  investigated.  Routine  use  of  modified  triangle  tests  should  lead  to 
experience  in  their  merits  relative  to  simple  triangle  tests  and  such  experience 
should  be  reported  in  subject  matter  literature, 

Bradley  [1963]  shows  the  association  between  triangle  tests  and  duo-trio 
tests.  A  modified  duo-trio  test  could  be  developed  in  just  the  same  way  as  for 
the  method  of  this  paper  for  the  modified  triangle  test  but  evidence  to  dat  e 
suggests  that  the  triangle  test  should  be  used  in  preference  to  the  duo- trio  test. 
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APPENDIX 

MOMENTS  OF  R  AND  W 

We  have  considered  first  and  second  moments  of  R  and  W  and  they  yield  very 
cumbersome  general  expressions.  However^  when  \i  (or  0)  is  zero,  these  expressions 
reduce  and  permit  an  estimate  of  a  that  may  be  used  in  the  iterative  procedures 
developed  for  maximum  likelihood  estimation  as  an  initial  trial  value. 

___  r\  O 

We  obtain  E(r|a,  =  0),  E(wjA,  H  =  0),  E(R  |a,  |i  =  0),  and  E(W  |a,  li  =  0) 
by  straight-forward  integration  after  polar  transformation  using  (7)  and  (10): 

2p^E(R|A,  p  =  0)  =  I^E(w|a,  11  =  0) 


75^:0 


iR  1 


•/2R  ^  -R'^/3a^ 


3a 


dR 


=  73  a/  t/jt  , 


2p^E(R^|A,  P  -  0)  =  P^E(W^|a,  =  0) 


dR 


a  [1  +  (373/2jr)]. 


(Al) 


(A2) 


Let  D  be  the  unconditional  random  variable  representing  degree  of  difference  so  that 
D  ~  R  when  the  triangle  test  is  correct  and  D  =  W  when  the  triangle  test  is  in¬ 


correct.  Then 


(A3) 
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e(d|  |1  =  0)  =  p^e(r|a,  li  =  0)  +  p^e(w|a,  h  =  o) 

=  3i/3  al^.  Vjt 
from  (Al) .  Similarly^ 

E(D^|p  =  0)  =  (3  a^/2)  +  (9^30^/  4rt)  (A4) 

from  (A2).  The  variance  of  given  p.  =  0,  in  the  usual  way  is 

.,V(D||i  =  0)  =  (30^/2)  -  (9Vla-/4jt)(73  -  1)  =  ,5919a^. 

An  initial  value  of  a  may  be  obtained  by  computation  of  the  sample  variance 

of  these  observations  taken  as  a  single  sample  of  size 

2  2 

liH-n*  If  this  sample  variance  is  s  ,  equate  s  to  V(d|^  =  0)  and  obtain 

=  s  /  •/.5919  .  (A5) 

It  appears  that  obtained  in  this  way  should  be  a  satisfactory  initial  estimate 
of  a  for  use  either  in  (17)  and  (18)  or  in  (20).  The  maximum  likelihood  estimates 
of  0  do  not  appear  to  differ  greatly  whether  or  not  0  is  taken  to  be  zero  in  the 
iterative  processes  and  in  general  a  is  only  a  little  greater  than  a. 


